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Abstract 

The purpose of this manual is to provide a gen- 
eral overview and basic planning information 
for the 580/Multiple Domain Feature (580/MDF). 

This publication assumes the reader is familiar 
with information contained in: 

■ IBM System/370 Principles of Operation 
(GA22-7000). 

■ IBM System/370 Extended Architecture 
Principles of Operation (SA22-7085). 

Additional information relating to the specifi- 
cation and operation of 580/MDF is available in 
the following Amdahl publications : 

" Amdahl 580 Computing Systems Computer 
Operator's Manual (MC- 108335). 

■ Amdahl 580 MACROCODE Reference 
Manual {UC- 119324). 
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Introduction 



The 580/Multiple Domain Feature 
(580/MDF) is an innovative way for 
580 users to manage and control their 
processing resources. 580/MDF, like 
the 580/ACCELERATOR, is another 
example of Amdahl's commitment 
to providing unique but compatible 
solutions to meet the needs of the 
large-systems user. 

580/MDF gives the 580 user the abil- 
ity to consolidate multiple computer 
systems into a single processing com- 
plex; in addition, when operating 
multiple system control programs 
(SCPs) on a single processor, the need 
to run VM is eliminated. (VM contin- 
ues to be supported on all 580 Series 
processors.) In addition, this 580 
Series hardware feature eliminates 
the need for independent production 
environments and separate test or 
conversion systems. Isolated, highly 
secure operation of multiple SCPs is 
available to the user without incur- 
ring the costs or limitations of owning 
and operating multiple processing 
complexes. 580/MDF provides 
multiple, independent operating 
environments, called domains, 
which are active concurrently on 
one 580 Series processor. 



580/MDF also provides the Amdahl 
580 Series with increased flexibility. 
A unique combination of hardware , 
microcode, and MACROCODE is 
used to support the definition, acti- 
vation, and operation of multiple 
operating environments in a single 
processor. 
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FIGURE 1 A Single Domain 




Why was 580/MDF 
developed? 

580/MDF was developed to provide 
the large-systems user the ability to 
operate more than one logical system 
on a single processor. The following 
580/MDF features help achieve this: 

■ Concurrent native support of 
System 370 (S/370) and 370 extended 
architecture (370-XA) modes 

■ No additional system control 
program (SCP) or other software 
required 

■ No required software 
modifications 

■ Redundant IBM software licenses 
may be eliminated 

■ Performance at least 95% of 
native mode 

■ Hardware isolation and 
protection for each domain 

■ Dynamic allocation and 
redistribution of CPU time 

■ Flexible allocation of main storage 
and channels from a pool of resources 

■ Operational characteristics of two 
physically separate computer systems 



What is a domain? 

An Amdahl 580 processor creates 
and maintains a processing environ- 
ment for the SCP. This environment 
consists of all the resources required 
to operate the SCP and is called a 
domain. 

The attributes of a domain include: 

■ architectural mode 

■ main storage size 

■ channel configuration 

■ operator facilities 

■ logical processor 



*A logical processor is functionally equivalent 
to a physical CPU. 
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FIGURE 2 Multiple Domains 




What is 580/HDF? 

580/MDF extends the domain concept 
one level further. It allows the user to 
define and operate multiple domains 
concurrently on a single 580 proces- 
sor. Each domain may be defined to 
operate in either S/370 or 370-XA 
mode, and represents a complete 
operating environment to the SCP. 

Operator interface 

580/MDF offers an easy-to-under- 
stand, easy-to-use operator interface. 
All domain definition and activation 
is menu driven and provides clear, 
explicit parameters to be entered at 
the 580/MDF master console. Gener- 
ally, domain definition is assigned 
to the 580 main operator console 
(MOC). A domain console is speci- 
fied for each domain definition. 
Although the functions of a domain 
console are a subset of the functions 
of the master console, Amdahl rec- 
ommends that a 580 remote operator 
console (ROC) be designated as the 
domain console. 

The logical processor is controlled 
at the domain console. The operator 
may start and stop the logical proces- 



sor, and inspect and alter registers 
or the program status word (PSW), 
while the logical processor in another 
domain is executing. The stop key on 
the console would, of course, stop the 
whole processor complex. The stop 
command at the domain console 
affects only the logical processor in 
that domain. IPL of the system con- 
trol program is also performed at 
the domain console. Operation of 
an SCP, after it has been IPLed, 
remains unchanged in a 580/MDF 
environment. 



What is 580/MDF? 



Resource Allocation 

Using an adjustable time-slicing tech- 
nique, 580/MDF gives control of the 
580 processor complex alternately to 
each one of the domains. In Figure 3, 
one domain is allocated 75 percent 
of the available CPU power. Each 
domain owns a portion of main 
storage and has exclusive access to 
a number of channels. CPU time can 
be dynamically redistributed either 
by the user or automatically by 
580/MDF. Main storage is allo- 
cated to each domain in 64K-byte 
units and must be contiguous. Indi- 
vidual channels are allocated to each 
domain and need not be contiguous. 
The 580 design allows each domain 
to remain absolutely secure. And, 
since domains cannot share main 
storage, a software error in one 
domain cannot affect another 
domain. 
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FIGURE 3 Resource Allocation 
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Storage Fencing 

Each domain is allocated storage 
from the total amount of main stor- 
age installed on the processor. A 
domain can access only its portion of 
main storage. There are no existing 
paths that allow a program in one 
domain to look into another domain's 
storage. The address range in each 
domain starts at zero. The domain 
addresses are mapped using a tech- 
nique which ensures data integrity 
in all domains. Whenever data are 
accessed in a domain, the real 
address is accompanied by a domain 
identification (ID). This ID serves as 
a pointer into an array of base and 
bound registers. If the sum of the real 
address and the contents of the base 
register is below the contents of the 
bound register, then access is 
granted. Unlike segment and page 
tables, the base and bound registers 
are in an area of main storage which 
is inaccessible from a domain. 

Channels access data in main storage 
and therefore carry a domain ID to 
ensure that they can only read from 
and write to the proper domain's 
storage. 
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FIGURE 4 Storage Fencing 



What is 580/MDF? 




FIGURE 5 Production and Test Domains 



Asynchronous Interrupts 

Some events occur asynchronously to 
CPU operations such as I/O and exter- 
nal interrupts. Both are logically 
associated with a certain domain and 
they must not interrupt a different 
domain. 

Therefore all I/O and certain external 
interrupts, such as timer interrupts, 
are associated with a domain ID. 
Both the I/O processor (IOP) and the 
memory bus controller (MBC), where 
the 580 interrupt routing facilities are 
located, are aware of which domain is 
currently in control. Any interrupt 
not matching the active domain's ID 
is kept pending in the hardware. 




How can 580/MDF 
be used? 

580/MDF provides the user with a 
flexible tool to address a variety of 
operational requirements such as: 

■ testing 

■ operational consolidation 

■ conversions 

■ backup strategies 



Testing 

580/MDF provides the user the ability 
to test SCPs, subsystems, and applica- 
tions during prime shift without 
requiring standalone processors. It 
also provides the user with the 
opportunity to conduct installation 
verification tests on new peripheral 
equipment in a timely and fail-safe 
fashion. SCPs and their subsystems, 
such as JES and VTAM, are difficult 
to test because they are fundamental 
to the operation — generally, testing 
must be conducted during off-shift 
hours on a standalone processor. 
With 580/MDF, the user can establish 
test environments to conduct this 
critical testing during prime time. 
Domain isolation ensures that SCP 
and subsystem failures are not propa- 
gated throughout the processing 
complex. 
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FIGURE 7 XA Conversion 
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FIGURE 8 A Backup Domain 



FIGURE 6 Two CPUs into One 



Operational Consolidation 

580/MDF allows data center manage- 
ment to combine several different 
processing requirements that are now 
operating on multiple computers to 
operate on one processor. When 
using multiple domains, the user may 
experience reduced operating costs, 
reduced software costs, and more 
efficient use of data processing 
resources. 



Conversions 

Most data processing installations 
face the ongoing requirement of 
conversions. Whether they are 
architectural, SCP, subsystem, or 
application conversions, 580/MDF 
offers an efficient facility for sup- 
porting the conversion process 
while continuing to run produc- 
tion workloads on the same 
processor without compromising 
the integrity of the production 
environment. 

Since the Amdahl 580 is capable of 
operating in either S/370 or 370/XA 
mode, 580/MDF allows each domain 
to operate in either mode without 
simulation. When migrating from 
MVS/370 to MVS/XA, for example, 
the XA domain may be allocated 
initially, e.g. , 8MB of main storage, 
8 channels, and 20 percent of CPU 
time. As the migration proceeds, and 
workloads shift to MVS/XA, more 
and more resources may be taken 
from the 370 domain and given to 
the XA domain. 



Backup 

An idle replica of a critical produc- 
tion system is operationally ideal, 
yet it is difficult to justify financially. 
While computer hardware is much 
more reliable today than a decade 
ago, networks have grown and there 
are many more potential single 
points of failure. 

An important enhancement to IMS 
is the Extended Recovery Facility 
(XRF). It is intended to improve 
availability "by using additional 
resources to lessen the impact of 
certain, events that disrupt service to 
the end users" (IBM announcement 
dated February 12, 1985). Access to 
multiple domains provides the user 
the ability to use a dormant domain 
to accommodate any recovery 
requirements. When XRF success- 
fully recovers, the dormant domain's 
share of the CPU resource is automat- 
ically increased. 



How can 580/MDF be used? 



2 Functional 

Summary 



This chapter discusses domain defi- 
nition, activation, and operation. 
Domain definition consists of specify- 
ing domain characteristics and saving 
the configuration for later use. 
Domain definition can be done at any 
time prior to activation. Domain acti- 
vation is the process used to enable a 
domain for operation. It consists of 
reading, or loading, the saved config- 
uration into the System Storage Area 
(SSA) and activating the domain. 
During domain activation each 
domain is allocated a subset of the 
total resources installed on the pro- 
cessor. Domain operation provides 
the basic operator facilities as defined 
in the IBM System! 370 Principles of 
Operation and the IBM System/370 
Extended Architecture Principles of 
Operation. 



Domain Definition 

During domain definition, the user 
specifies the characteristics of the 
domain. As stated before, these 
characteristics include: 

■ architectural mode 

■ main storage size 

■ channel configurations 

■ operator facilities 

■ logical processor 

The user may save the domain defini- 
tion for later recall and use. Specific 
information about the use of 
MACROCODE and the command 
frames can be found in the Amdahl 
' 580 MACROCODE Reference 
Manual. 
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FIGURE 9 Configuration Attributes Frame 



Architectural Mode 

To set the mode of a domain, specify 
either 370 or XA on the configuration 
attributes (CA) frame. 



Domain Definition 
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FIGURE10 Main Storage Allocation 



Main Storage Size 

Main storage is allocated to a domain 
by specifying the domain's main stor- 
age size on the CA frame. MACRO- 
CODE requires 768K bytes of main 
storage with or without 580/MDF 
installed. (For detailed information 
on 580 main storage, refer to the 580 
MACROCODE Reference Manual.) 

Domain Console Address 

The configuration I/O (CI) frame 
specifies the logical address of the 
580 console to be used as the domain 
console. The domain console must 
be a MOC or a ROC. It is used for all 
domain specific control functions 
such as START, STOP, and LOAD 
(IPL). 
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FIGURE 11 Index Frame 
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FIGURE 12 Channel Configuration 



Channel Allocation 

580 system channels are allocated to 
a domain, and their logical addresses 
are established through information 
supplied on the CI frame or, if one or 
more 370-XA domains are to be acti- 
vated, in the input output control 
data set (IOCDS). Devices can be 
shared in the same way they are 
shared between separate physical 
processors. 

CPU Time Allocation 

580/MDF's flexible time-slicing tech- 
nique is also used to make CPU time 
available to each domain. For each 
domain, the operator specifies two 
parameters: the target value (this 
is the percentage of CPU time the 
domain should receive) and the 
maximum value (MAX- value). The 
MAX-value establishes an upper limit 
on the percentage of CPU time the 
domain may use. In figure 13, a target 
value of 80 percent is specified for the 
production domain and a target value 
of 20 percent for the test domain. 



If the test domain does not use its 
allotted CPU time, the production 
domain is allowed to use all available 
CPU time. In this case, the produc- 
tion domain has its MAX-value 
specified at 100 percent. On the other 
hand, if the production domain is less 
active, then the test domain is limited 
to 50 percent of the available CPU 
time. Such a setup would guarantee 
that important users, even if they 
are temporarily idle, always have 
sufficient CPU time allotted to 
access the system. 




FIGURE 13 CPU Target and MAX-Values 



Domain Definition 
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The CPU resource allocation objec- 
tives are specified on the system 
scheduling (SS) frame. Since 
response time interacts with through- 
put time, the user may control overall 
performance through the scheduling 
parameter. This parameter deter- 
mines the frequency of each domain 
receiving control (for a full descrip- 
tion, refer to "System Programming 
Considerations" in this document). 
The user can select the value which 
best meets the installation's require- 
ments. The value can be dynamically 
changed. 
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Saving the Configuration 

The configuration created during 
domain definition can be saved for 
later recall and use. The config- 
uration is saved by selecting the 
configuration control (CC) frame, 
naming the domain, and selecting 
the"S" option. 
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FIGURE 15 Configuration Control Frame 



Domain Definition 
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USER INTERFACE 
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FIGURE 16 Domain Consoles 



Domain Activation 

Domain activation is the process by 
which resources are allocated to the 
domain according to the domain con- 
figuration specifications. The domain 
configuration is read and loaded into 
the system storage area (SSA) by 
selecting the CC frame, entering the 
name of the domain or configuration, 
and using the "R" option. The 
domain may then be activated by 
specifying the "A" option on the 
CC frame. 



Modifying the Configuration 

The domain configuration may be 
modified during the definition, acti- 
vation, and operation phases. The 
CPU resource allocation objectives 
can be changed on any domain 
without interrupting operation. 

Channels may be dynamically 
reallocated from one domain to 
another provided both domains 
are operating in the same architec- 
tural mode. It is the user's respon- 
sibility to ensure that the channels 
affected are varied offline and that 
they are inactive before they are 
reallocated. 

Reconfiguration of domain storage, 
or changing the architectural mode of 
a domain, requires deactivation of 
the domain, redefinition of the 
storage size or architectural mode, 
reactivation of the domain, and 
re-IPLingtheSCPs. 




Domain Operation 

The primary interface between the 
user and 580/MDF is the domain 
console. 

In case of a console failure, the sys- 
tem automatically initiates a console 
switch. The SCP is IPLed from a 
domain console and operation 
proceeds in the usual manner 
from any available SCP console. 
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Planning 
Information 



System Control 
Program Support 

580/MDF supports the following 
SCP environments: 

■ MVS/SP Version 1, Release 3 
(MVS/370) 

■ MVS/SP Version 2, Release 1 
(MVS/XA) 

■ VM/SP High Performance Option 
(VM/SP HPO) Releases 3.2 and 3.4 

For support of later versions and 
releases of these SCPs, please refer 
to the current Amdahl Software 
Support announcement. 



SCP Consoles 

Any terminal which is supported by 
the SCP may be used as an SCP mas- 
ter console. It is also possible to use a 
580 MOC or ROC as an SCP console, 
but this option may be chosen for no 
more than one active domain. 
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Trace and Debug 

The trace facilities provided by 
the SCPs are available for use 
with 580/MDF. 

Debugging facilities provided by the 
580 Series processors are available at 
the domain console or the 580/MDF 
master console . All of these facilities 
affect only one domain. Available 
facilities include: 

■ ALTER/DISPLAY of domain 
storage 

■ ALTER/DISPLAY of program 
status word 

■ ALTER/DISPLAY of control 
registers 

■ ALTER/DISPLAY of general 
purpose registers 

■ ALTER/DISPLAY of floating 
point registers 

■ set address compare 

■ START/STOP 

■ instruction step 

■ store status 
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System Programming 
Considerations 

Performance Aspects 

580/MDF has flexible tuning parame- 
ters to give the user maximum control 
over the performance of each 
domain. The user can explicitly 
define the amount of CPU time that 
can be used by each domain by speci- 
fying the same percentages as the 
target and MAX-values established 
during domain definition (see "CPU 
Time Allocation" section of this 
document for more information) . 

Performance of a domain in a 580/ 
MDF environment is expected to be 
near that experienced in a single 
domain environment with the same 
resources. When a domain is given 
control, it has exclusive control of the 
processor and the resources allocated 
to it. Given the same resources, mul- 
tiple domains typically provide at 
least 95 percent of the performance 
experienced with 580/MDF installed. 
Performance experienced when 



operating only one domain with 580/ 
MDF installed is equivalent to that 
experienced without 580/MDF 
installed. 

The time-slicing technique is used 
to allocate the CPU resource to the 
domains. The scheduling parameter 
determines the duration of the time 
slice. This parameter influences the 
number of domain switches. The 
scheduling parameter is specified on 
the SS frame as an integer between 
1 and 5 (default = 3). When a higher 
number is specified, the frequency 
of each domain receiving control is 
raised, favoring response-time- 
oriented work. When a lower 
number is specified, each domain 
stays in control for a longer period 
of time, and unnecessary domain 
switches are avoided. Thus, the 
user can balance performance of the 
system to achieve optimum through- 
put and responsiveness. 

Performance, throughput, and trans- 
action rates depend on configuration, 
applications, operating characteris- 
tics, and the CPU resource objectives 
specified during domain definition. 
Individual user environments should 
be carefully evaluated before making 
any performance estimates. 



Timing Considerations 

All timing facilities described in the 
System/370 Principles of Operations and 
the System/370 Extended Architecture 
Principles of Operations are available 
in each domain. Both the time-of-day 
(TOD) clock and the clock comparator 
are based on elapsed (wallclock) time. 
Both time facilities are provided for 
each domain independently. The TOD 
clock continues to run when the domain 
is not in control. Therefore, the TOD 
clock should be used with caution when 
monitoring an individual domain. 

There is a CPU timer facility for each 
domain. This timer is only active when 
the domain is in control. In an MVS sys- 
tem, all TCB and SRB times, as well as 
the accumulated wait times in RMF, are 
collected by the dispatcher using the 
CPU timer. All TCB, SRB, and Wait 
times shown in the RMF reports as 
seen by that domain are correct. 



System Programming Considerations 



17 



All percentages reported by a moni- 
tor in one domain relate to the pro- 
portion of the CPU time that domain 
has been using — not to the whole 
processor. When two domains report 
70 percent and 80 percent CPU busy 
respectively, this does not mean that 
the Amdahl 580 processor was 150 
percent busy. To calculate the per- 
cent of the time the total CPU was 
busy: 



Total CPU busy 



elapsed time - (domainl wait time + domain2 wait time) 



elapsed time 



I/O Statistics 



All I/O counts are correct. Once an 
I/O operation is started, it executes 
asynchronously and it may end when 
the initiating domain is not in con- 
trol. In this case the interrupt is kept 
pending until the domain is in control 
again. The time between these events 
(Start I/O and interrupt) is reported 
as the device response time. Even if 
the interrupt is held pending, the I/O 
operation completes in its usual 
amount of time. However, in 370- 
mode, I/O completion is detected 
only when the interrupt is taken, so 
the reported device response time 



may seem to be higher. In XA-mode, 
all I/O statistics are collected in the 
I/O processor, regardless of the 
domain in control. The time the 
MVS/XA system spends in interrupt 
handling is not included in the 
device response time, as is the case 
when operating in MVS/370 mode. 

Sharing Data Sets 

All of the provisions that are required 
to ensure data integrity when sharing 
data sets among multiple processors 
must also be taken when sharing data 
sets among multiple domains. These 
provisions include that the devices 
to be shared are properly declared 
in the system generation. 

CPU ID 

The CPU ID is returned by the STIDP 
instruction. It contains information 
identifying the domain number. 
Refer to the Appendix for complete 
information on the layout and content 
of this data area. 
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IOCDS Considerations 

Only one IOCDS is necessary when 
operating multiple 370-XA domains. 
The IOCDS should include devices 
that are currently installed as well as 
those planned for the complex. This 
reduces the frequency of input out- 
put control program (IOCP) genera- 
tions. During domain activation, the 
configuration described in the IOCDS 
is used as a resource pool for alloca- 
tion to the domains. 

EREP Reports 

History data on systemwide errors 
are centrally maintained in the con- 
sole subsystem and stored in the 
MOC. In multiprocessor systems, 
each MOC stores the data pertinent 
to that side of the processor complex 
to which the MOC is attached. The 
Amdahl-supplied Environmental 
Recording, Editing, and Printing 
Program (AMDEREP) obtains these 
data by accessing an OSI device (for a 
full description refer to the Amdahl 
580 Computing Systems Computer 
Operator's Manual). Therefore, 
AMDEREP should be run in the 
domain that has the OSI devices 
attached to it. 



Inter-Domain 
Communication 

Inter-domain communication sup- 
ports the same facilities currently 
available for connecting two physical 
processors. These include: shared 
DASD, channel-to-channel adapters, 
and teleprocessing links. 



Feature Support 

580/ACCELERATOR 

The 580/ACCELERATOR feature 
provides the capability to increase 
the performance level of an Amdahl 
5840 or 5850 processor. The 580/ 
ACCELERATOR feature is sup- 
ported by 580/MDF. All domains are 
accelerated equally when the 580/ 
ACCELERATOR feature is active. 



Feature Support 
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System Activity Monitor (SAM) 

SAM is a standard feature on all 580 
Series processors. It collects and 
graphically displays system utiliza- 
tion data. It is supported with 580/ 
MDF installed and provides infor- 
mation for individual domains as 
well as the entire processor. 

Hardware Monitor 
Attachment Feature fHMAF) 

HM AF provides for attachment of 
customer-owned hardware perfor- 
mance monitoring equipment to a 
580 Series processor. HMAF is sup- 
ported with 580/MDF installed and 
provides information for individual 
domains as well as the entire 
processor. 

High Speed Floating 
Point (HSFP) Feature 

The HSFP feature provides additional 
computing capacity for applications 
which use the floating point arithme- 
tic instruction set. The HSFP feature 
is supported with 580/MDF installed. 



M 



FIGURE 18 System Activity Monitor 
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580/MDF 



Appendix 



The CPU ID that is stored for an indi- 
vidual domain is a double word con- 
taining five fields of information. The 
first byte of the first word is the envi- 
ronment/version code. Bits through 
3 are used to indicate the operating 
environment of the processor and the 
presence of features. Bits 4 through 7 
contain the encoded model number. 
Bits 8 through 11 pertain to multipro- 
cessors and domains. Bits 12 through 
15 provide the CPU ID from the first 
two digits of the processor complex's 
serial number. Bits 16 through 31 are 
the low order four digits of the serial 
number. The second word contains 
two constants: the machine series 
(X'0580'), bits 32 through 47, and the 
Machine Check Extended Logout 
length (X'0000'), bits 48 through 63. 

The contents of the double word 
returned by the STIDP instruction 
are outlined below. 
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VERSION CODE 


CPU ADDRESS 


CPU ID NUMBER 


EVFAMMMM 


PDDL 


XZZZZ 


7 8 11 


12 31 


SERIES NUMBER 


MCEL 


0580 


0000 



32 



47 48 



63 



FIGURE 19 Results of stidp Instruction 



LEGEND FOR FIGURE 19 (ABOVE): 
Version Code (EVFA) 



BitO 
Bit 1 

Bit 2 

Bit 3 



(E)-BT 
(V)-BT 
(F)-B'T 
(A) — BT 



macrocode installed 
Running under VM 
hsfp feature installed 
580/ACCELERATOR feature installed 



Model Number of Complex (MMMM) 



Bits 4-7 
B'0101' 
B'0110' 
B'0010' 

B'0111' 
B'1000' 
B'0011' 
B'0100' 

CPU Address (pddl) 



— X'5' 

— X'6' 

— X'2' 

— X'7' 

— X'8' 

— X'3' 

— X'4' 



5840 
5850 
5860 
5867 
5868 
5870 
5880 



Bit 8 (P)-BT 

Bits 9-10 (DD) 



Bit 11 



(L)- 



-B'OT 

-B'O' 
-B'1' 



Side B of partitioned multiprocessor 
Domain number 

Domain 

Domain 1 
Logical processor number 

LPO 

LP1 



CPU Identification Number (XZZZZ) 



Bits 12-15 (X) — X'O' High order digit of the processor complex serial number. 

Bits 1 6-31 (ZZZZ) — X'0000' The low order 4 digits of the processor complex serial number. 
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